Week7 day 2 - Spatial Data / spatial analysis
What do you think spatial data looks like? Latitude and longitude
What do you think spatial data visualisation looks like?
Spatial (geospatial data) is data about geographical locations
How is spatial data represented?
Spatial data are special - there are meaning to the points; while encoded as numbers (dbl’s), these numbers actually have a special meaning
Spatial vectors (non-R specific)
Spatial vectors encode spatial data - there are 3 main types
- point (1, 4)
- line ((1,4), (4,5))
- polygon ((2,2), (4,2), (5,3), etc…)
point; train stations, wells, etc.. lines; roads.. polygons; council areas, lakes
Spatial data in R
Sometimes we just get given data frames with lat/long columns
Another option is to load in shapefiles
to do this, we use the package sf - simplefeatures
library(sf)
Linking to GEOS 3.9.1, GDAL 3.2.3, PROJ 7.2.1
a feature = a geometry
north_carolina <- st_read(system.file("shape/nc.shp", package = "sf"))
Reading layer `nc' from data source
`/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/sf/shape/nc.shp'
using driver `ESRI Shapefile'
Simple feature collection with 100 features and 14 fields
Geometry type: MULTIPOLYGON
Dimension: XY
Bounding box: xmin: -84.32385 ymin: 33.88199 xmax: -75.45698 ymax: 36.58965
Geodetic CRS: NAD27
library(tidyverse)
Registered S3 methods overwritten by 'dbplyr':
method from
print.tbl_lazy
print.tbl_sql
── Attaching packages ─────────────────────────────────────────────────────────────────── tidyverse 1.3.1 ──
✓ ggplot2 3.3.5 ✓ purrr 0.3.4
✓ tibble 3.1.5 ✓ dplyr 1.0.7
✓ tidyr 1.1.4 ✓ stringr 1.4.0
✓ readr 2.0.2 ✓ forcats 0.5.1
── Conflicts ────────────────────────────────────────────────────────────────────── tidyverse_conflicts() ──
x dplyr::filter() masks stats::filter()
x dplyr::lag() masks stats::lag()
north_carolina %>%
as_tibble()
nc_geo <- st_geometry(north_carolina)
nc_geo[[1]]
MULTIPOLYGON (((-81.47276 36.23436, -81.54084 36.27251, -81.56198 36.27359, -81.63306 36.34069, -81.74107 36.39178, -81.69828 36.47178, -81.7028 36.51934, -81.67 36.58965, -81.3453 36.57286, -81.34754 36.53791, -81.32478 36.51368, -81.31332 36.4807, -81.26624 36.43721, -81.26284 36.40504, -81.24069 36.37942, -81.23989 36.36536, -81.26424 36.35241, -81.32899 36.3635, -81.36137 36.35316, -81.36569 36.33905, -81.35413 36.29972, -81.36745 36.2787, -81.40639 36.28505, -81.41233 36.26729, -81.43104 36.26072, -81.45289 36.23959, -81.47276 36.23436)))
class(north_carolina)
[1] "sf" "data.frame"
it’s a dataframe with geometries
Plotting geometries
plot(north_carolina["AREA"])

# first row, "AREA" column
plot(north_carolina[1, "AREA"])

Task - Have a look through some of the variables within your north_carolina dataset, and see if you can create a spatial plot using the techniques above.
plot(north_carolina["FIPS"])

ggplot and sf
ggplot can plot spatial with sf, called geom_sf
north_carolina %>%
ggplot() +
geom_sf(aes(fill = SID74), colour = "black") +
theme_bw()

Task - try plotting another feature for NC. What does it tell you? play around with ggplot aspects (e.g. colour)
north_carolina %>%
ggplot() +
geom_sf(aes(fill = FIPS), colour = "black", show.legend = FALSE) +
theme_void() +
ylim(20, 50)

NA
library(rgeos)
Loading required package: sp
rgeos version: 0.5-8, (SVN revision 679)
GEOS runtime version: 3.9.1-CAPI-1.14.2
Please note that rgeos will be retired by the end of 2023,
plan transition to sf functions using GEOS at your earliest convenience.
GEOS using OverlayNG
Linking to sp version: 1.4-5
Polygon checking: TRUE
library(rnaturalearth)
library(rnaturalearthdata)
Using rnaturalearth package we can import boundaries of countries at various levels
world <- ne_countries(scale = "medium", returnclass = "sf")
world %>%
as_tibble() %>%
head(5)
So we’ve got the data, now we can plot it
world %>%
ggplot() +
geom_sf() +
labs(x = "longitude", y = "latitude", title = "World Map")

We can plot actual data now:
world %>%
ggplot() +
geom_sf(aes(fill = pop_est)) +
scale_fill_viridis_c(trans = "sqrt")

Task: Recap your knowledge from ggplot week, and set your geom_sf aesthetic to be filled with the estimated gdp (gdp_md_est variable). Extra points if you make your map colour blind friendly! What does your plot tell you? What does it tell you compared to the population?
world %>%
ggplot() +
geom_sf(aes(fill = gdp_md_est)) +
scale_fill_continuous(type = "viridis") +
theme_void()

# It tells me that while China and India are greater than the rest of the world
# in terms of population, the US is greater than the China and India in terms
# of GDP (despite having lower population)
With our world data frame, we can filter for specific countries, using the tidyverse
country_italy <- world %>%
filter(name == "Italy")
# plot just italy
country_italy %>%
ggplot() +
geom_sf() +
labs(x = "Longitude",
y = "Latitude",
title = "Italy")

country_denmark <- world %>%
filter(name == "Denmark")
country_denmark %>%
ggplot() +
geom_sf() +
labs(x = "Longitude",
y = "Latitude",
title = "Danmark")

country_faroes <- world %>%
filter(name == "Faeroe Is.")
country_faroes%>%
ggplot() +
geom_sf() +
labs(x = "Longitude",
y = "Latitude",
title = "Færøerne")

Zooming in on particular parts of the world
- we can subset our graph by limiting the x and y range using coord_sf()
world %>%
ggplot() +
geom_sf() +
coord_sf(xlim = c(-102.15, -74.12), ylim = c(7.65, 33.97), expand = FALSE)

Add informative labels
- we need to tell ggplot where to put the label
- therefore we need to calculate where we want to put the labels (centre of each feature)
- we need to calculate the centres (centroids)
world %>%
mutate(centre = st_centroid(st_make_valid(geometry))) %>%
as_tibble() %>%
select(name, centre)
world_with_centres <- world %>%
mutate(centre = st_centroid(st_make_valid(geometry))) %>%
mutate(lat = st_coordinates(centre)[,1],
long = st_coordinates(centre)[,2])
# centre geometries
# purely illustrative
centre_geo <- world %>%
mutate(centre = st_centroid(st_make_valid(geometry))) %>%
select(centre)
centre_geo[[1]][[1]]
POINT (-69.98266 12.52088)
world_with_centres %>%
ggplot() +
geom_sf() +
geom_text(aes(x = lat, y = long, label = name), colour = "darkblue",
fontface = "bold", check_overlap = TRUE, size = 3) +
coord_sf(xlim = c(-102.15, -74.12), ylim = c(7.65, 33.97), expand = FALSE)

We can add additional information using annotate
- add one that says: Gulf of Mexico
world_with_centres %>%
ggplot() +
geom_sf() +
geom_text(aes(x = lat, y = long, label = name), colour = "darkblue",
fontface = "bold", check_overlap = TRUE, size = 3) +
annotate(geom = "text", x = -90, y = 26, label = "Gulf of Mexico", size = 5,
fontface = "italic")+
coord_sf(xlim = c(-102.15, -74.12), ylim = c(7.65, 33.97), expand = FALSE)

Task
world_with_centres %>%
ggplot() +
geom_sf(aes(fill = income_grp)) +
geom_text(aes(x = lat, y = long, label = name), colour = "black",
fontface = "bold", check_overlap = TRUE, size = 3) +
annotate(geom = "text", x = -90, y = 26, label = "Gulf of Mexico", size = 3,
fontface = "italic")+
coord_sf(xlim = c(-110.15, -60.12), ylim = c(2.65, 50.97), expand = TRUE) +
labs(x = "Latitude",
y = "Longitude",
fill = "OECD Income group")

Interactive maps with leaflet
library(leaflet)
Registered S3 method overwritten by 'htmlwidgets':
method from
print.htmlwidget tools:rstudio
leaflet() %>%
addTiles() %>% #basemap
addMarkers(lng = 174.768, lat = -36.852, popup = "The birthplace of R")
That was our (maybe) first leaflet
Get some spatial data from the web - turn it into a format R can work with - visualise it using leaflet
library(jsonlite)
Attaching package: ‘jsonlite’
The following object is masked from ‘package:purrr’:
flatten
colorado_data_url <-
"https://data.colorado.gov/resource/j5pc-4t32.json?&county=BOULDER"
head(readLines(colorado_data_url))
Warning in readLines(colorado_data_url) :
incomplete final line found on 'https://data.colorado.gov/resource/j5pc-4t32.json?&county=BOULDER'
[1] "[ {"
[2] " \"station_name\" : \"PECK-PELLA AUGMENTATION RETURN\","
[3] " \"div\" : \"1\","
[4] " \"location\" : {"
[5] " \"latitude\" : \"40.160705\","
[6] " \"needs_recoding\" : false,"
jsonlite package has a lot of functions to help work with json data
json is like a list of lists in R - sometimes we need to recursively sift through to extract the relevant data
colorado_water <- fromJSON(colorado_data_url) %>%
jsonlite::flatten(recursive = TRUE)
# wrangling
colorado_water_clean <- colorado_water %>%
select(-location.needs_recoding) %>%
mutate(across(c(starts_with("location"), amount), as.numeric)) %>%
filter(!is.na(location.latitude), !is.na(location.longitude))
# visualise with leaflet
colorado_water_clean %>%
leaflet() %>%
addTiles() %>%
addCircleMarkers(lng = ~location.longitude,
lat = ~location.latitude)
- incidences of surface water in and around Boulder, Colorado
colorado_water_clean %>%
leaflet() %>%
addTiles() %>%
addCircleMarkers(lng = ~location.longitude,
lat = ~location.latitude,
radius = ~log(amount), weight = 1)
Warning in log(amount) : NaNs produced
Clustering
Let’s have a look at what addMarkers looks like if we have lots of points
colorado_water_clean %>%
leaflet() %>%
addTiles() %>%
addMarkers(lng = ~location.longitude,
lat = ~location.latitude)
We can add clustering options
colorado_water_clean %>%
leaflet() %>%
addTiles() %>%
addMarkers(lng = ~location.longitude,
lat = ~location.latitude,
clusterOptions = markerClusterOptions())
Leaflet in shiny
maybe too many? allow user to filter for specific regions only view distilleries in that region
whisky_df %>%
leaflet() %>%
addTiles() %>%
addCircleMarkers(lat = ~lat, lng = ~long, popup = ~Distillery)
LS0tCnRpdGxlOiAiUiBOb3RlYm9vayIKb3V0cHV0OiBodG1sX25vdGVib29rCi0tLQoKV2VlazcgZGF5IDIgLSBTcGF0aWFsIERhdGEgLyBzcGF0aWFsIGFuYWx5c2lzCgpXaGF0IGRvIHlvdSB0aGluayBzcGF0aWFsIGRhdGEgbG9va3MgbGlrZT8KICBMYXRpdHVkZSBhbmQgbG9uZ2l0dWRlCiAgCiAgCldoYXQgZG8geW91IHRoaW5rIHNwYXRpYWwgZGF0YSB2aXN1YWxpc2F0aW9uIGxvb2tzIGxpa2U/CiAgClNwYXRpYWwgKGdlb3NwYXRpYWwgZGF0YSkgaXMgZGF0YSBhYm91dCBnZW9ncmFwaGljYWwgbG9jYXRpb25zCgojIyBIb3cgaXMgc3BhdGlhbCBkYXRhIHJlcHJlc2VudGVkPwoKClNwYXRpYWwgZGF0YSBhcmUgc3BlY2lhbCAtIHRoZXJlIGFyZSBtZWFuaW5nIHRvIHRoZSBwb2ludHM7IHdoaWxlIGVuY29kZWQKYXMgbnVtYmVycyAoZGJsJ3MpLCB0aGVzZSBudW1iZXJzIGFjdHVhbGx5IGhhdmUgYSBzcGVjaWFsIG1lYW5pbmcKClNwYXRpYWwgdmVjdG9ycyAobm9uLVIgc3BlY2lmaWMpCgpTcGF0aWFsIHZlY3RvcnMgZW5jb2RlIHNwYXRpYWwgZGF0YSAtIHRoZXJlIGFyZSAzIG1haW4gdHlwZXMKCi0gcG9pbnQgKDEsIDQpCi0gbGluZSAoKDEsNCksICg0LDUpKQotIHBvbHlnb24gKCgyLDIpLCAoNCwyKSwgKDUsMyksIGV0Yy4uLikKCnBvaW50OyB0cmFpbiBzdGF0aW9ucywgd2VsbHMsIGV0Yy4uCmxpbmVzOyByb2Fkcy4uCnBvbHlnb25zOyBjb3VuY2lsIGFyZWFzLCBsYWtlcwoKIyMgU3BhdGlhbCBkYXRhIGluIFIKClNvbWV0aW1lcyB3ZSBqdXN0IGdldCBnaXZlbiBkYXRhIGZyYW1lcyB3aXRoIGxhdC9sb25nIGNvbHVtbnMKCkFub3RoZXIgb3B0aW9uIGlzIHRvIGxvYWQgaW4gc2hhcGVmaWxlcwoKdG8gZG8gdGhpcywgd2UgdXNlIHRoZSBwYWNrYWdlIHNmIC0gc2ltcGxlZmVhdHVyZXMKCmBgYHtyfQpsaWJyYXJ5KHNmKQpgYGAKCmEgZmVhdHVyZSA9IGEgZ2VvbWV0cnkKCmBgYHtyfQpub3J0aF9jYXJvbGluYSA8LSBzdF9yZWFkKHN5c3RlbS5maWxlKCJzaGFwZS9uYy5zaHAiLCBwYWNrYWdlID0gInNmIikpCmBgYAoKYGBge3J9CmxpYnJhcnkodGlkeXZlcnNlKQpgYGAKCgpgYGB7cn0Kbm9ydGhfY2Fyb2xpbmEgJT4lCiAgYXNfdGliYmxlKCkKYGBgCgpgYGB7cn0KbmNfZ2VvIDwtIHN0X2dlb21ldHJ5KG5vcnRoX2Nhcm9saW5hKQoKbmNfZ2VvW1sxXV0KYGBgCgpgYGB7cn0KY2xhc3Mobm9ydGhfY2Fyb2xpbmEpCmBgYAppdCdzIGEgZGF0YWZyYW1lIHdpdGggZ2VvbWV0cmllcwoKCiMjIFBsb3R0aW5nIGdlb21ldHJpZXMKCgpgYGB7cn0KcGxvdChub3J0aF9jYXJvbGluYVsiQVJFQSJdKQpgYGAKCmBgYHtyfQojIGZpcnN0IHJvdywgIkFSRUEiIGNvbHVtbgpwbG90KG5vcnRoX2Nhcm9saW5hWzEsICJBUkVBIl0pCmBgYAoKVGFzayAtIEhhdmUgYSBsb29rIHRocm91Z2ggc29tZSBvZiB0aGUgdmFyaWFibGVzIHdpdGhpbiB5b3VyIG5vcnRoX2Nhcm9saW5hIGRhdGFzZXQsIGFuZCBzZWUgaWYgeW91IGNhbiBjcmVhdGUgYSBzcGF0aWFsIHBsb3QgdXNpbmcgdGhlIHRlY2huaXF1ZXMgYWJvdmUuCmBgYHtyfQpwbG90KG5vcnRoX2Nhcm9saW5hWyJGSVBTIl0pCmBgYAojIyBnZ3Bsb3QgYW5kIHNmCgpnZ3Bsb3QgY2FuIHBsb3Qgc3BhdGlhbCB3aXRoIHNmLCBjYWxsZWQgZ2VvbV9zZgoKYGBge3J9Cm5vcnRoX2Nhcm9saW5hICU+JQogIGdncGxvdCgpICsKICBnZW9tX3NmKGFlcyhmaWxsID0gU0lENzQpLCBjb2xvdXIgPSAiYmxhY2siKSArCiAgdGhlbWVfYncoKQpgYGAKClRhc2sgLSB0cnkgcGxvdHRpbmcgYW5vdGhlciBmZWF0dXJlIGZvciBOQy4gV2hhdCBkb2VzIGl0IHRlbGwgeW91PyBwbGF5IGFyb3VuZAp3aXRoIGdncGxvdCBhc3BlY3RzIChlLmcuIGNvbG91cikKYGBge3J9Cm5vcnRoX2Nhcm9saW5hICU+JQogIGdncGxvdCgpICsKICBnZW9tX3NmKGFlcyhmaWxsID0gRklQUyksIGNvbG91ciA9ICJibGFjayIsIHNob3cubGVnZW5kID0gRkFMU0UpICsKICB0aGVtZV92b2lkKCkgKwogIHlsaW0oMjAsIDUwKQogIApgYGAKCmBgYHtyfQpsaWJyYXJ5KHJnZW9zKQpsaWJyYXJ5KHJuYXR1cmFsZWFydGgpCmxpYnJhcnkocm5hdHVyYWxlYXJ0aGRhdGEpCmBgYAoKVXNpbmcgcm5hdHVyYWxlYXJ0aCBwYWNrYWdlIHdlIGNhbiBpbXBvcnQgYm91bmRhcmllcyBvZiBjb3VudHJpZXMgYXQgdmFyaW91cwpsZXZlbHMKCmBgYHtyfQp3b3JsZCA8LSBuZV9jb3VudHJpZXMoc2NhbGUgPSAibWVkaXVtIiwgcmV0dXJuY2xhc3MgPSAic2YiKQpgYGAKCmBgYHtyfQp3b3JsZCAlPiUKICBhc190aWJibGUoKSAlPiUKICBoZWFkKDUpCmBgYAoKU28gd2UndmUgZ290IHRoZSBkYXRhLCBub3cgd2UgY2FuIHBsb3QgaXQKCmBgYHtyfQp3b3JsZCAlPiUKICBnZ3Bsb3QoKSArCiAgZ2VvbV9zZigpICsKICBsYWJzKHggPSAibG9uZ2l0dWRlIiwgeSA9ICJsYXRpdHVkZSIsIHRpdGxlID0gIldvcmxkIE1hcCIpCmBgYAoKV2UgY2FuIHBsb3QgYWN0dWFsIGRhdGEgbm93OiAKCmBgYHtyfQp3b3JsZCAlPiUKICBnZ3Bsb3QoKSArCiAgZ2VvbV9zZihhZXMoZmlsbCA9IHBvcF9lc3QpKSArCiAgc2NhbGVfZmlsbF92aXJpZGlzX2ModHJhbnMgPSAic3FydCIpCmBgYAoKVGFzazogUmVjYXAgeW91ciBrbm93bGVkZ2UgZnJvbSBnZ3Bsb3Qgd2VlaywgYW5kIHNldCB5b3VyIGdlb21fc2YgYWVzdGhldGljIHRvIGJlIGZpbGxlZCB3aXRoIHRoZSBlc3RpbWF0ZWQgZ2RwIChnZHBfbWRfZXN0IHZhcmlhYmxlKS4gRXh0cmEgcG9pbnRzIGlmIHlvdSBtYWtlIHlvdXIgbWFwIGNvbG91ciBibGluZCBmcmllbmRseSEKV2hhdCBkb2VzIHlvdXIgcGxvdCB0ZWxsIHlvdT8gV2hhdCBkb2VzIGl0IHRlbGwgeW91IGNvbXBhcmVkIHRvIHRoZSBwb3B1bGF0aW9uPwoKYGBge3J9CndvcmxkICU+JQogIGdncGxvdCgpICsKICBnZW9tX3NmKGFlcyhmaWxsID0gZ2RwX21kX2VzdCkpICsKICBzY2FsZV9maWxsX2NvbnRpbnVvdXModHlwZSA9ICJ2aXJpZGlzIikgKwogIHRoZW1lX3ZvaWQoKQoKIyBJdCB0ZWxscyBtZSB0aGF0IHdoaWxlIENoaW5hIGFuZCBJbmRpYSBhcmUgZ3JlYXRlciB0aGFuIHRoZSByZXN0IG9mIHRoZSB3b3JsZAojIGluIHRlcm1zIG9mIHBvcHVsYXRpb24sIHRoZSBVUyBpcyBncmVhdGVyIHRoYW4gdGhlIENoaW5hIGFuZCBJbmRpYSBpbiB0ZXJtcwojIG9mIEdEUCAoZGVzcGl0ZSBoYXZpbmcgbG93ZXIgcG9wdWxhdGlvbikKYGBgCgpXaXRoIG91ciB3b3JsZCBkYXRhIGZyYW1lLCB3ZSBjYW4gZmlsdGVyIGZvciBzcGVjaWZpYyBjb3VudHJpZXMsIHVzaW5nIHRoZSAKdGlkeXZlcnNlCgpgYGB7cn0KY291bnRyeV9pdGFseSA8LSB3b3JsZCAlPiUKICBmaWx0ZXIobmFtZSA9PSAiSXRhbHkiKQpgYGAKCmBgYHtyfQojIHBsb3QganVzdCBpdGFseQoKY291bnRyeV9pdGFseSAlPiUKICBnZ3Bsb3QoKSArCiAgZ2VvbV9zZigpICsKICBsYWJzKHggPSAiTG9uZ2l0dWRlIiwKICAgICAgIHkgPSAiTGF0aXR1ZGUiLAogICAgICAgdGl0bGUgPSAiSXRhbHkiKQpgYGAKCgpgYGB7cn0KY291bnRyeV9kZW5tYXJrIDwtIHdvcmxkICU+JQogIGZpbHRlcihuYW1lID09ICJEZW5tYXJrIikKCmNvdW50cnlfZGVubWFyayAlPiUKICBnZ3Bsb3QoKSArCiAgZ2VvbV9zZigpICsKICBsYWJzKHggPSAiTG9uZ2l0dWRlIiwKICAgICAgIHkgPSAiTGF0aXR1ZGUiLAogICAgICAgdGl0bGUgPSAiRGFubWFyayIpCgpjb3VudHJ5X2Zhcm9lcyA8LSB3b3JsZCAlPiUKICBmaWx0ZXIobmFtZSA9PSAiRmFlcm9lIElzLiIpCgpjb3VudHJ5X2Zhcm9lcyU+JQogIGdncGxvdCgpICsKICBnZW9tX3NmKCkgKwogIGxhYnMoeCA9ICJMb25naXR1ZGUiLAogICAgICAgeSA9ICJMYXRpdHVkZSIsCiAgICAgICB0aXRsZSA9ICJGw6Zyw7hlcm5lIikKCmBgYAoKIyMgWm9vbWluZyBpbiBvbiBwYXJ0aWN1bGFyIHBhcnRzIG9mIHRoZSB3b3JsZAoKLSB3ZSBjYW4gc3Vic2V0IG91ciBncmFwaCBieSBsaW1pdGluZyB0aGUgeCBhbmQgeSByYW5nZSB1c2luZyBjb29yZF9zZigpCgpgYGB7cn0Kd29ybGQgJT4lCiAgZ2dwbG90KCkgKwogIGdlb21fc2YoKSArCiAgY29vcmRfc2YoeGxpbSA9IGMoLTEwMi4xNSwgLTc0LjEyKSwgeWxpbSA9IGMoNy42NSwgMzMuOTcpLCBleHBhbmQgPSBGQUxTRSkKYGBgCgpBZGQgaW5mb3JtYXRpdmUgbGFiZWxzCgotIHdlIG5lZWQgdG8gdGVsbCBnZ3Bsb3Qgd2hlcmUgdG8gcHV0IHRoZSBsYWJlbAotIHRoZXJlZm9yZSB3ZSBuZWVkIHRvIGNhbGN1bGF0ZSB3aGVyZSB3ZSB3YW50IHRvIHB1dCB0aGUgbGFiZWxzIChjZW50cmUgb2YgCmVhY2ggZmVhdHVyZSkKLSB3ZSBuZWVkIHRvIGNhbGN1bGF0ZSB0aGUgY2VudHJlcyAoY2VudHJvaWRzKQoKYGBge3J9CndvcmxkICU+JQogIG11dGF0ZShjZW50cmUgPSBzdF9jZW50cm9pZChzdF9tYWtlX3ZhbGlkKGdlb21ldHJ5KSkpICU+JQogIGFzX3RpYmJsZSgpICU+JQogIHNlbGVjdChuYW1lLCBjZW50cmUpCmBgYAoKYGBge3J9CndvcmxkX3dpdGhfY2VudHJlcyA8LSB3b3JsZCAlPiUKICBtdXRhdGUoY2VudHJlID0gc3RfY2VudHJvaWQoc3RfbWFrZV92YWxpZChnZW9tZXRyeSkpKSAlPiUKICBtdXRhdGUobGF0ID0gc3RfY29vcmRpbmF0ZXMoY2VudHJlKVssMV0sCiAgICAgICAgIGxvbmcgPSBzdF9jb29yZGluYXRlcyhjZW50cmUpWywyXSkKYGBgCgoKYGBge3J9CiMgY2VudHJlIGdlb21ldHJpZXMKIyBwdXJlbHkgaWxsdXN0cmF0aXZlCgpjZW50cmVfZ2VvIDwtIHdvcmxkICU+JQogIG11dGF0ZShjZW50cmUgPSBzdF9jZW50cm9pZChzdF9tYWtlX3ZhbGlkKGdlb21ldHJ5KSkpICU+JQogIHNlbGVjdChjZW50cmUpCmBgYAoKCmBgYHtyfQpjZW50cmVfZ2VvW1sxXV1bWzFdXQpgYGAKCgpgYGB7cn0Kd29ybGRfd2l0aF9jZW50cmVzICU+JQogIGdncGxvdCgpICsKICBnZW9tX3NmKCkgKwogIGdlb21fdGV4dChhZXMoeCA9IGxhdCwgeSA9IGxvbmcsIGxhYmVsID0gbmFtZSksIGNvbG91ciA9ICJkYXJrYmx1ZSIsCiAgICAgICAgICAgIGZvbnRmYWNlID0gImJvbGQiLCBjaGVja19vdmVybGFwID0gVFJVRSwgc2l6ZSA9IDMpICsKICBjb29yZF9zZih4bGltID0gYygtMTAyLjE1LCAtNzQuMTIpLCB5bGltID0gYyg3LjY1LCAzMy45NyksIGV4cGFuZCA9IEZBTFNFKQpgYGAKCldlIGNhbiBhZGQgYWRkaXRpb25hbCBpbmZvcm1hdGlvbiB1c2luZyBhbm5vdGF0ZSAKCi0gYWRkIG9uZSB0aGF0IHNheXM6IEd1bGYgb2YgTWV4aWNvCgpgYGB7cn0Kd29ybGRfd2l0aF9jZW50cmVzICU+JQogIGdncGxvdCgpICsKICBnZW9tX3NmKCkgKwogIGdlb21fdGV4dChhZXMoeCA9IGxhdCwgeSA9IGxvbmcsIGxhYmVsID0gbmFtZSksIGNvbG91ciA9ICJkYXJrYmx1ZSIsCiAgICAgICAgICAgIGZvbnRmYWNlID0gImJvbGQiLCBjaGVja19vdmVybGFwID0gVFJVRSwgc2l6ZSA9IDMpICsKICBhbm5vdGF0ZShnZW9tID0gInRleHQiLCB4ID0gLTkwLCB5ID0gMjYsIGxhYmVsID0gIkd1bGYgb2YgTWV4aWNvIiwgc2l6ZSA9IDUsCiAgICAgICAgICAgZm9udGZhY2UgPSAiaXRhbGljIikrCiAgY29vcmRfc2YoeGxpbSA9IGMoLTEwMi4xNSwgLTc0LjEyKSwgeWxpbSA9IGMoNy42NSwgMzMuOTcpLCBleHBhbmQgPSBGQUxTRSkgCmBgYAoKVGFzawpgYGB7cn0Kd29ybGRfd2l0aF9jZW50cmVzICU+JQogIGdncGxvdCgpICsKICBnZW9tX3NmKGFlcyhmaWxsID0gaW5jb21lX2dycCkpICsKICBnZW9tX3RleHQoYWVzKHggPSBsYXQsIHkgPSBsb25nLCBsYWJlbCA9IG5hbWUpLCBjb2xvdXIgPSAiYmxhY2siLAogICAgICAgICAgICBmb250ZmFjZSA9ICJib2xkIiwgY2hlY2tfb3ZlcmxhcCA9IFRSVUUsIHNpemUgPSAzKSArCiAgYW5ub3RhdGUoZ2VvbSA9ICJ0ZXh0IiwgeCA9IC05MCwgeSA9IDI2LCBsYWJlbCA9ICJHdWxmIG9mIE1leGljbyIsIHNpemUgPSAzLAogICAgICAgICAgIGZvbnRmYWNlID0gIml0YWxpYyIpKwogIGNvb3JkX3NmKHhsaW0gPSBjKC0xMTAuMTUsIC02MC4xMiksIHlsaW0gPSBjKDIuNjUsIDUwLjk3KSwgZXhwYW5kID0gVFJVRSkgKwogIGxhYnMoeCA9ICJMYXRpdHVkZSIsCiAgICAgICB5ID0gIkxvbmdpdHVkZSIsCiAgICAgICBmaWxsID0gIk9FQ0QgSW5jb21lIGdyb3VwIikKYGBgCgojIyBJbnRlcmFjdGl2ZSBtYXBzIHdpdGggbGVhZmxldAoKYGBge3J9CmxpYnJhcnkobGVhZmxldCkKYGBgCgoKYGBge3J9CmxlYWZsZXQoKSAlPiUKICBhZGRUaWxlcygpICU+JSAjYmFzZW1hcAogIGFkZE1hcmtlcnMobG5nID0gMTc0Ljc2OCwgbGF0ID0gLTM2Ljg1MiwgcG9wdXAgPSAiVGhlIGJpcnRocGxhY2Ugb2YgUiIpCmBgYAoKVGhhdCB3YXMgb3VyIChtYXliZSkgZmlyc3QgbGVhZmxldAoKR2V0IHNvbWUgc3BhdGlhbCBkYXRhIGZyb20gdGhlIHdlYgotIHR1cm4gaXQgaW50byBhIGZvcm1hdCBSIGNhbiB3b3JrIHdpdGgKLSB2aXN1YWxpc2UgaXQgdXNpbmcgbGVhZmxldAoKYGBge3J9CmxpYnJhcnkoanNvbmxpdGUpCmBgYAoKYGBge3J9CmNvbG9yYWRvX2RhdGFfdXJsIDwtCiAgImh0dHBzOi8vZGF0YS5jb2xvcmFkby5nb3YvcmVzb3VyY2UvajVwYy00dDMyLmpzb24/JmNvdW50eT1CT1VMREVSIgpgYGAKCmBgYHtyfQpoZWFkKHJlYWRMaW5lcyhjb2xvcmFkb19kYXRhX3VybCkpCmBgYAoKanNvbmxpdGUgcGFja2FnZSBoYXMgYSBsb3Qgb2YgZnVuY3Rpb25zIHRvIGhlbHAgd29yayB3aXRoIGpzb24gZGF0YQoKanNvbiBpcyBsaWtlIGEgbGlzdCBvZiBsaXN0cyBpbiBSIC0gc29tZXRpbWVzIHdlIG5lZWQgdG8gcmVjdXJzaXZlbHkgc2lmdCB0aHJvdWdoCnRvIGV4dHJhY3QgdGhlIHJlbGV2YW50IGRhdGEKCmBgYHtyfQpjb2xvcmFkb193YXRlciA8LSBmcm9tSlNPTihjb2xvcmFkb19kYXRhX3VybCkgJT4lCiAganNvbmxpdGU6OmZsYXR0ZW4ocmVjdXJzaXZlID0gVFJVRSkKYGBgCgpgYGB7cn0KIyB3cmFuZ2xpbmcKY29sb3JhZG9fd2F0ZXJfY2xlYW4gPC0gY29sb3JhZG9fd2F0ZXIgJT4lCiAgc2VsZWN0KC1sb2NhdGlvbi5uZWVkc19yZWNvZGluZykgJT4lCiAgbXV0YXRlKGFjcm9zcyhjKHN0YXJ0c193aXRoKCJsb2NhdGlvbiIpLCBhbW91bnQpLCBhcy5udW1lcmljKSkgJT4lCiAgZmlsdGVyKCFpcy5uYShsb2NhdGlvbi5sYXRpdHVkZSksICFpcy5uYShsb2NhdGlvbi5sb25naXR1ZGUpKQogIApgYGAKCmBgYHtyfQojIHZpc3VhbGlzZSB3aXRoIGxlYWZsZXQKCmNvbG9yYWRvX3dhdGVyX2NsZWFuICU+JQogIGxlYWZsZXQoKSAlPiUKICBhZGRUaWxlcygpICU+JQogIGFkZENpcmNsZU1hcmtlcnMobG5nID0gfmxvY2F0aW9uLmxvbmdpdHVkZSwKICAgICAgICAgICAgICAgICAgIGxhdCA9IH5sb2NhdGlvbi5sYXRpdHVkZSkKYGBgCgotIGluY2lkZW5jZXMgb2Ygc3VyZmFjZSB3YXRlciBpbiBhbmQgYXJvdW5kIEJvdWxkZXIsIENvbG9yYWRvCgpgYGB7cn0KY29sb3JhZG9fd2F0ZXJfY2xlYW4gJT4lCiAgbGVhZmxldCgpICU+JQogIGFkZFRpbGVzKCkgJT4lCiAgYWRkQ2lyY2xlTWFya2VycyhsbmcgPSB+bG9jYXRpb24ubG9uZ2l0dWRlLAogICAgICAgICAgICAgICAgICAgbGF0ID0gfmxvY2F0aW9uLmxhdGl0dWRlLAogICAgICAgICAgICAgICAgICAgcmFkaXVzID0gfmxvZyhhbW91bnQpLCB3ZWlnaHQgPSAxKQpgYGAKCiMjIENsdXN0ZXJpbmcKCkxldCdzIGhhdmUgYSBsb29rIGF0IHdoYXQgYWRkTWFya2VycyBsb29rcyBsaWtlIGlmIHdlIGhhdmUgbG90cyBvZiBwb2ludHMKCmBgYHtyfQpjb2xvcmFkb193YXRlcl9jbGVhbiAlPiUKICBsZWFmbGV0KCkgJT4lCiAgYWRkVGlsZXMoKSAlPiUKICBhZGRNYXJrZXJzKGxuZyA9IH5sb2NhdGlvbi5sb25naXR1ZGUsCiAgICAgICAgICAgICAgICAgICBsYXQgPSB+bG9jYXRpb24ubGF0aXR1ZGUpCmBgYAoKV2UgY2FuIGFkZCBjbHVzdGVyaW5nIG9wdGlvbnMKCmBgYHtyfQpjb2xvcmFkb193YXRlcl9jbGVhbiAlPiUKICBsZWFmbGV0KCkgJT4lCiAgYWRkVGlsZXMoKSAlPiUKICBhZGRNYXJrZXJzKGxuZyA9IH5sb2NhdGlvbi5sb25naXR1ZGUsCiAgICAgICAgICAgICAgICAgICBsYXQgPSB+bG9jYXRpb24ubGF0aXR1ZGUsCiAgICAgICAgICAgICBjbHVzdGVyT3B0aW9ucyA9IG1hcmtlckNsdXN0ZXJPcHRpb25zKCkpCmBgYAoKIyMgTGVhZmxldCBpbiBzaGlueQoKYGBge3J9CndoaXNreV9kZiA8LSBDb2RlQ2xhbkRhdGE6OndoaXNreSAlPiUKICByZW5hbWUobG9uZyA9IExhdGl0dWRlLAogICAgICAgICBsYXQgPSBMb25naXR1ZGUpCmBgYAoKbWF5YmUgdG9vIG1hbnk/CmFsbG93IHVzZXIgdG8gZmlsdGVyIGZvciBzcGVjaWZpYyByZWdpb25zCm9ubHkgdmlldyBkaXN0aWxsZXJpZXMgaW4gdGhhdCByZWdpb24KCmBgYHtyfQp3aGlza3lfZGYgJT4lCiAgbGVhZmxldCgpICU+JQogIGFkZFRpbGVzKCkgJT4lCiAgYWRkQ2lyY2xlTWFya2VycyhsYXQgPSB+bGF0LCBsbmcgPSB+bG9uZywgcG9wdXAgPSB+RGlzdGlsbGVyeSkKYGBgCgo=